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We prove that all subclasses of the separable permutations not 
■^J- ' containing Av(231) or a symmetry of this class have rational gen- 

erating functions. Our principal tools are partial well-order, atom- 
icity, and the theory of strongly rational permutation classes intro- 
duced here for the first time. 



1. INTRODUCTION 



The separable permutations are those which can be built from the permutation 1 by repeat- 
edly applying two operations, known as direct sum (or simply, sum) and skew sum which 
are defined, respectively, on permutations it of length m and a of length n by 

(7T©(T)(«) = 

(nQa)(i) = 

In this introductory section, we recapitulate some known results about the separable per- 
mutations and some related sets of permutations. The operations © and are best under- 
stood by considering the plots of the permutations, as in Figure 1. 



7T(«) 


if 1 < i < m, 


cr(i — m) + m 


\im + \<i<m + n, 


n(i) + n 


if 1 < i < m, 


a{i — m) 


ifm + l<i<7n + n, 
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Figure 1: An example of a direct sum and a skew sum. 



While the term "separable permutation" dates only to the work of Bose, Buss, and 
Lubiw [6], these permutations first arose in Avis and Newborn's work on pop stacks [5]. 
Separable permutations are the permutation analogues of two other well-studied classes 
of object: complement-reducible graphs (also called cographs, or simply P^free graphs), 
and series-parallel (or A-free) posets. A folkloric result, which follows from the character- 
izations of these analogous classes, characterizes the separable permutations. 

Proposition 1.1. A permutation it is separable if and only if it contains neither 2413 nor 3142. 

In Proposition 1.1, we say that the permutation it of length n contains the permutation 
a of [k] (written a < it) if it has a subsequence of length k order isomorphic to a. For 
example, it = 89167342 (written in list, or one-line notation) contains a = 51342, as can be 
seen by considering the subsequence 91672. 

Our interest is with sets of permutations which are closed downward under this con- 
tainment order, which we call permutation classes (or occasionally just classes). Thus, C is a 
class if for all it in C and all a < it, a is also in C. One way to specify classes is as closures: 
if X is any set of permutations, its closure is the permutation class 

C\(X) = {a : a < it for some it G X}. 

However, it is often more useful to specify classes by what they do not contain; for any 
permutation class C there is a unique (and possibly infinite) antichain B such that 

C = Av(B) = {vr : vr ^ /3 for all /3 € B}. 

This antichain B is called the basis of C, so the basis of the class of separable permutations 
is {2413,3142}. Another way to characterize the separable permutations is provided by 
the next result; in this result we write C © T> for the set of permutations of the form it © a 
where it lies in C and a lies in V, and extend this definition to C V analogously. (If C and 
V are permutation classes, then so are C © V and C © V.) 
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Proposition 1.2. The class of separable permutations is the smallest nonempty class C which sat- 
isfies both C®C CCandCeC C C. 

Throughout this paper, S will be used to denote the class of separable permutations. 
Any class which satisfies C © C C C is called sum closed, while any class satisfying CqC CC 
is called skew closed. 

The separable permutations contain a notable subclass, Av(231), which Knuth [11] 
showed are precisely the permutations that can be sorted by a stack (a last-in first-out 
list). A result similar to Proposition 1.2 holds for Av(231) as well: 

Proposition 1.3. The class Av(231) is the smallest nonempty class C which satisfies both C(BC C C 
andlQC CC. 

In fact, since both basis elements (2413 and 3142) of S each contain every non-monotone 
permutation of length 3, all four of the classes Av(132), Av(213), Av(231), and Av(312) 
are contained in S, and each of these has a characterisation similar to the one given by 
Proposition 1.3. These four classes are all symmetric images of one another under the 
operations of reversal, inverse, and complementation (or compositions of these), each of 
which preserves the containment order. 

For any class C (or more generally any set of permutations), we denote by C n the set 
of permutations in C of length n, and say that the generating function for C is J2 \C n \x n - 
Whether this sum includes the empty permutation (n = 0) is a matter of taste and conve- 
nience, and we generally elect not to include it. 

Note that every separable permutation of length at least 2 — and by extension, every 
permutation in Av(231) of length at least 2 — is either sum decomposable, meaning that it is 
equal to tt © a for two shorter (but nonempty) permutations tt and a, or it is skew decompos- 
able, which is defined analogously. No permutation is both sum and skew decomposable, 
so the separable permutations may therefore be partitioned into three sets: {1}, the sum 
decomposable separable permutations, and the skew decomposable separable permuta- 
tions. This observation allows one to easily enumerate the class. 

Proposition 1.4. The generating function for the separable permutations is 

1 — x — yl — 6x + x 2 
2 

and thus the number of separable permutations of length n is the n th large Schroder number. 

Proof. Let / denote the generating function for the class of separable permutations, /® 
the generating function for its sum decomposable elements, and / e the generating func- 
tion for its skew decomposable elements. As observed above, we have / = x + /© + /q. 
Any sum decomposable permutation may be written uniquely as the direct sum of a sum 
indecomposable permutation and another permutation, so since the class of separable per- 
mutations is sum closed, we have /© = (/ — /©)/■ Solving this shows that /© = / 2 /(l + /), 
and then by symmetry / e = f 2 /(l + /), so / = x + 2/ 2 /(l + /). Solving this yields the 
desired generating function. □ 
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A similar approach gives the generating function for Av(231): 
Proposition 1.5. The generating function for Av (231) is 



1 - 2x - VI -4x 
2x 

and thus the number of ' 231-avoiding permutations of length n is the n th Catalan number. 

Proof. Let / denote the generating function for Av(231), and let /® and / e count the sum 
and skew decomposable permutations in this class. Since Av(231) is sum closed, by the 
same logic as in the proof of Proposition 1.4, f 9 = / 2 /(l + /). Now note that ir Q a G 
Av(231) if and only if it is decreasing and a G Av(231). Thus every skew decomposable 
permutation in Av(231) may be written uniquely as 1 a for a G Av(231), so f Q = xf. 
Substituting these values into our equation / = x + /© + /e yields that / = x(l + f) 2 , and 
solving this gives the generating function claimed. □ 

Note that both of these generating functions are nonrational. Clearly we cannot hope 
for the generating function of a generic superclass C D Av(231) to be rational (although 
this may happen by accident). Our main result establishes the converse: if C is a subclass 
of the separable permutations that does not contain any of Av(132), Av(213), Av(231) or 
Av(312), then C has a rational generating function. 

2. Partial Well-Order and Atomicity 

Many of our arguments depend on the partial well-order (pzvo) property. In the context of the 
containment order on permutations, a permutation class has the pwo property if it does 
not contain an infinite antichain. This property has the following well-known consequence 
which is important to us because it allows us to consider minimal counterexamples within 
a pwo class. 

Proposition 2.1. The subclasses of a pwo class C satisfy the minimum condition, i.e., every family 
of subclasses of a pwo class C has a minimal subclass under inclusion. 

Proof If there were a family of subclasses with no minimal element then we could, induc- 
tively, find a strictly descending chain 

C 1 DC 2 2 ••• 

of subclasses of C. For each i > 1, choose /% G C % \ C' L+1 . The set of minimal elements of 
{/3i, (32 ■ ■ •} is an antichain and therefore finite. Hence there exists an integer n such that 
{/3i, /?2 • • • , /3n} contains these minimal elements. In particular /3 m < /3 n +i for some m <n, 
but C n+1 is a class, and therefore f3 m G C n+1 C C m+1 , a contradiction. □ 

For any class C, we define its sum completion C as the smallest sum closed class con- 
taining C, and we define its strong completion, sc(C), as the smallest © and skew sum closed 
class containing C. In this notation [2, Theorem 2.5] states 
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Proposition 2.2. The sum completion and the strong completion of a pwo class are pwo. 

Since, clearly, the separable permutations are the strong completion of the set {1} we 
see that the separable class is pwo. 

Another key concept that we shall require is atomicity. A permutation class is called 
atomic if it cannot be written as the union of two proper subclasses. The notion of atomicity 
was first studied (in a more general context) by Frai'sse [10], who established several alter- 
native characterizations of this property. The only characterization we require features in 
our next proposition. For a proof of this result in the context of permutations, we refer to 
Atkinson, Murphy and Ruskuc [3, Theorem 1.2]. 

Proposition 2.3. If C is an atomic class then there is a chain ct\ < a>2 < ■ ■ ■ of permutations in C 
such that C = C\({a 1 ,a 2 , ■••})• 

We refer to such a chain as a spine for the class. 

3. Strongly Rational Classes 

Our main goal is to prove that if a subclass of the separable permutations does not contain 
Av(231) or any of its symmetries, then it and all of its subclasses have rational generating 
functions. In this section we study this powerful property in its own right, beginning by 
naming it: the permutation class C is strongly rational if it and all of its subclasses have 
rational generating functions. While strongly rational classes are naturally defined and 
appear to be the "correct" context in which to state and prove the tools of this section, they 
have received virtually no attention before, and many conjectures remain. 

Proposition 3.1. The union and intersection of two strongly rational classes is strongly rational. 

Proof The intersection of two strongly rational classes is contained in both of them, and so 
strongly rational by definition. Now suppose that C and T> are strongly rational and that 
£ C C UV. Since £ = (C n £) U (V n £), we can enumerate it by inclusion-exclusion; the 
generating function for £ is the generating function for C n £ plus the generating function 
for V n £ minus the generating function for C n V n £. As C and V are strongly rational, all 
of these generating functions are rational, so £ has a rational generating function, verifying 
that C U V is strongly rational. □ 

We note that Proposition 3.1 does not hold for classes with rational generating func- 
tions in general. Neither does our next proposition, which follows from an argument of 
Atkinson and Stitt [4] first formalized by Murphy [13, Chapter 9] (although not in this 
context). 

Proposition 3.2. Strongly rational classes are partially well ordered. 

Proof. Suppose that the class C is not pwo. Therefore it contains an infinite antichain, and 
in particular contains an infinite antichain A C C with at most one member of each length. 
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If A\ / A2 are two subsets of A, then the two subclasses C n Av(Al) and C n Av(^.2) have 
different enumerations. To see this, suppose that a of length k is the shortest permutation 
in one but not both of A\ and A^. Then C n Av(Ai) and C n Av(^2) contain the same 
permutations of length less than k but differ by one in the number of permutations of 
length k. 

Because A is infinite, it follows that C has uncountably many subclasses with different 
generating functions. These generating functions cannot all be rational, so C is not strongly 
rational. □ 

Our next step on the path to more powerful tools is the following. 

Proposition 3.3. If the class C is strongly rational, then each of the sets of its sum indecomposable 
permutations, its sum decomposable permutations, its skew indecomposable permutations and its 
skew decomposable permutations have rational generating functions. 

Proof. It suffices to prove the claim for the sum indecomposable permutations in C as the 
remaining cases follow by symmetry or subtraction. If the claim were false then, because 
strongly rational classes are pwo by Proposition 3.2, the minimum condition of Proposi- 
tion 2.1 shows that any counterexample would have a minimal subclass that was also a 
counterexample. Choose C to be such a minimal counterexample. By Proposition 2.2 C 
is pwo and so the antichain of minimal elements of the difference (© C) \C is a finite 
set, say {/3i, . . . , j3 m }. Clearly the /% are nothing other than the sum decomposable basis 
elements of C. Suppose that 

A = /8j,i e A,2 e • • • e &,„, 

where the fyjS are sum indecomposable. Now, for any permutation 717 let b(-7r) = (bx, . . . , b m ) 
where for each i, ir contains /^i © • • • © /^ but avoids /^i © • • • © /^ © Pi t b i+1 . Note that 
b(-7r) < n — 1 = (m — 1, ... , n m — 1) for all permutations it € C. (Here and in what follows 
the order < on vectors is the dominance order, meaning that (pi, . . . ,p m ) < (iXj ■ ■ ■ > Qm) if 
and only if pi < qi for all 1 < i < m.) 

We now define a variety of generating functions: 

• / denotes the generating function for the class C, 

• for a vector p of natural numbers, / p denotes the generating function for all permu- 
tations in C which avoid Pi m +x © • • • © fii,m f° r a U h 

• /® denotes the generating function for the sum decomposable permutations in C, 

• f$ denotes the generating function for the sum indecomposable permutations in C, 
and 

• for a vector p of natural numbers, /5 denotes the generating function for the sum 
indecomposable permutations in C with b(7r) = p. 
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Note that f@, the generating function we wish to prove rational, is the sum of the 
generating functions /5 for all < p < n — 1. We now claim that ifO<p<n— 1 
then /? is rational. We establish this claim by induction on the sum of the entries of p. It 
is clearly true for the base case /£, as this function counts sum indecomposable elements 
of the proper subclass {ir € C : h(ir) = 0} of C. For larger p, /2 can be expressed as 
the difference between the generating function for sum indecomposable elements of the 
proper subclass {it G C : b(7r) < p} of C (which is rational by our choice of C) and the sum 
of the generating functions /3 for all < q < p (which are rational by induction). This 
claim established, our goal is only to show that /m _1 is rational. 

Now we aim to enumerate C, thereby expressing /, which is known to be rational, in 
terms of the /£ and f p functions. In the resulting equation f^ 1 will be the only term not 
already known to be rational, yielding a contradiction and completing the proof. 

Consider the sum decomposable permutations of C. Each of these can be expressed 
uniquely as 8 © (j) where 6 is indecomposable and so, counting how many of the summands 
in fii are contained in 6, we see that / e is the sum of f&fp for all vectors 0<p<n — 1. Of 
course, f$ is the sum of the generating functions /£ and so we obtain the equation 

/ = /* + /© = £ /J+ £ /J/ P = £ /g(/p + i) 

0<P<n-l 0<p<n-l 0<p<n-l 



Isolating the final term on the right hand side shows: 

/;- 1 (/n-l + l) = /- £ /JUp + 1)- 
0<p<n-l 

As we have previously remarked, every generating function on the right-hand side and 
in the second factor of the left hand side is rational, so /m _1 and thus f$ must be rational 
as well. This contradiction to our choice of C completes the proof. □ 

It would be possible at this point to use Proposition 3.3 to prove that if C is strongly 
rational then C is strongly rational as well. Instead, we provide a more powerful tool 
which we need for the main theorem. 

A permutation is said to be skew-merged if it is the union of an increasing subsequence 
and a decreasing subsequence. The class of skew-merged permutations was first studied 
by Stankova [14] in one of the earliest papers on permutation patterns, and later enumer- 
ated by Atkinson [1]. Stankova proved that the skew-merged permutations have the basis 
{2143, 3412}, a result which can also be seen to follow from Foldes and Hammer's char- 
acterization of split graphs [9]. Our interest lies with the class of separable skew-merged 
permutations, 

X = Av(2143, 2413, 3142, 3412). 

We label this class X because, in his thesis, Waton [15] showed that these are precisely 
the permutations that can be "drawn on an X" via the following procedure: choose — from 
an X made of right angles which form 45° angles with the axes in the plane — n points, no 
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Figure 2: The permutation 7253461 can be drawn on an X. 

two lying on a common vertical or horizontal line, and label these points 1, . . . , n reading 
bottom-to-top, then record these values reading left-to-right, as depicted in Figure 2. We 
may also define the class X in a manner similar to Propositions 1.2 and 1.3. 

Proposition 3.4. The class X is the smallest nonempty class C which contains C © 1, 1 © C, C © 1 
and 1 © C. 

Waton enumerated the class X, obtaining the generating function (1 — 3x)/(l — 4x + 
2x 2 ). Later, Elizalde [8] constructed a bijection between the class X and the set of "almost- 
increasing permutations" considered by Knuth [12, Section 5.4.8, Exercise 8]. 

For the proof of our main result, we are interested not in the class X but rather in the 
Af-inflation of a strongly rational class U. This inflation, denoted X[U], can be visualised by 
taking any permutation in X , drawing it on the X as above, and then replacing each point 
in this drawing with a set of points corresponding to a permutation in IA in such a way 
that the relationships between elements belonging to different points (of the permutation 
from X) are the same as those between the original points. Thus each point on the original 
drawing is "inflated" into a permutation from U. As we show below, such inflations are 
strongly rational. Note, as made explicit in Corollary 3.6, that this theorem implies that the 
sum completion of a strongly rational class is again strongly rational. 

Theorem 3.5. IfU is a strongly rational class then X[U] is also strongly rational. 

Proof. Let Ube a strongly rational class. It is instructive to first consider the enumeration 
of X[U] itself. Given a sum decomposable permutation in X[U], it may decompose in one 
of two ways, either as a member of (U@) © X [U], or as a member of X [U] © (U$), or both, 
where U$ denotes the set of sum indecomposable elements of U. The intersection of these 
two sets is (U$) © {Af [£/] U e} © (U$), where e denotes the empty permutation. Doing the 
same for skew decomposable elements of X [U] leads us to the equation 

9 = x + 2f g - fl(g + 1) + 2f g - f*(g + 1), (1) 

where g denotes the generating function for X[U], f$ the generating function for 14$, and 
/0 the generating function for U^ . Solving for g shows that it is indeed rational in f$ and 
/0, which are themselves elements of Q(x) by Proposition 3.3. Specifically, 

x ~ fgk~ foi 



1-2/0+^-2/0 + /, 



2 • 
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Reassuringly, substituting f$ = f@ = x gives us the generating function, (x — 2x 2 )/(l — 
4x + 2x 2 ), which, upon adding 1 to count the empty permutation, agrees with Waton's 
enumeration of X = X[l\. 

In order to complete the proof — that all subclasses of X [U] have rational generating 
functions — we adapt some notation of Brignall, Huczynska, and Vatter [7]. A -property is 
any set of permutations, and we say that tt satisfies the property P if ir G P. Given a set 
of properties V , we say that V is separable query-complete if, for all nonempty permutations 
a and r (not necessarily lying in any class) and P G V, it can be decided whether <r ffi r 
and a Q t satisfy P given only the knowledge about what properties in V are satisfied by 
a and r. For example, letting ffi denote the set of sum decomposable permutations, we 
see that {ffi} is trivially separable complete: assuming that a and r are nonempty, a © r 
always satisfies © while a © r never satisfies ffi. Also note that for any permutation /3, the 
set {Av(5) : S < /3} is separable query-complete: a© r lies in Av(<5) if and only if a G Av(7) 
or r G Av(t) for all 7, i < S < ft satisfying 7 © 1 = 5. 

Returning to the situation at hand, consider an arbitrary subclass V C X\U]. As U 
is strongly rational it is pwo by Proposition 3.2. Thus X\U] is contained in the strong 
completion of a pwo class, and so is itself pwo by Proposition 2.2. Hence X[U]\V has only 
a finite number of minimal elements. This set B of minimal elements completely defines 
P as a subclass of X [U] because X[U] \T> is the set of permutations of X [U] that contain 
one or more permutations of B. It follows that {Av(<5) : 5 G Cl(B)} is a finite separable 
query-complete set, since it is the union of a finite number of separable query-complete 
properties. Slightly more generally, 

7? = {©,©}u{Av(S) :SeCl(B)} 

is also a finite separable query-complete set of properties. 

For any permutation tt, let 'P(vr) denote the set of properties in V satisfied by n. We 
introduce three families of generating functions which are defined for any subset QCP: 

• /g, the generating function for the set {tt G U : V(tt) = Q}, 

• 9Q> the generating function for the set {it G X[U] : V{tt) 5 Q}, 

• \iq, the generating function for the set {ir G X[U] : V(tt) = Q}. 

Our goal, with this notation, is to show that all functions of the form qq are rational, since 
it will then follow that the generating function for V, namely <7{Av(/3):/3gB}' i s rational. First, 
note that all of the /g generating functions are rational, by the strong rationality of U and 
Proposition 3.3. Also note that <?g is the sum of all h-n with Q C 1Z C V. Thus it will suffice 
to establish that the /ig generating functions are rational. 

We are now ready to describe the analogues of the terms of (1) which relate the h gener- 
ating functions to the / generating functions and to each other. Consider any subset QCp 
of properties containing ©. The permutations satisfying Q must be sum decomposable, 
and thus can be expressed as a ffi r where at least one of a or r is a sum indecomposable 
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permutation in U. Following our derivation of (1), such a Hq can be expressed as a linear 
combination of terms of three forms: 

• fnh-s with @ $lH, which count permutations from (U$) © X[C], 

• hsfr with © ^ T, which count permutations from X [U] © (U@), and 

• fnhsfr with © ^TZ,T, which count permutations from (U@) © X[U] © (14$), 

the latter occurring with negative coefficients to correct for over-counting. Similarly, if 
QCP contains 0, then fiQ can be expressed as a linear combination of terms of the form 
fnhs with © £ 1Z, hsfr with © ^ T, and fnhsfr with © ^ 1Z,T. As no permutation can 
be both sum decomposable and sum indecomposable there is only one more case, where 
neither © nor © lie in Q. However, in this case the only permutations in X\U] that can 
satisfy precisely the properties Q are those of C, so here Kq = /g. 

Therefore, letting h denote the column vector consisting of the Kq generating functions, 
there is some matrix M of rational functions in Q(x) and some constant vector v over 
Q(x) such that h = Mh + v. Since all of our generating functions enumerate non-empty 
permutations, the entries of M all have zero constant term. Hence I — M is invertible over 
Q(x), and thus each entry of h is a rational function, proving the theorem. □ 

Corollary 3.6. IfU is a strongly rational class then 0£Y is also strongly rational, and ifV and £ 
are strongly rational, then V © £ is also strongly rational. 

Proof. Since 0^/ C X[U], the first part of the corollary follows immediately from Theo- 
rem 3.5. For the second part of the corollary, Proposition 3.1 shows that T> U £ is strongly 
rational, and so (V U £ ) is also strongly rational. However V © £ C (V U £) complet- 
ing the proof. □ 

4. Proof of the Main Result 

The machinery of Sections 2 and 3 developed, we are now ready to state and prove our 
main result. 

Theorem 4.1. IfC is a subclass of the separable permutations that does not contain any of Av(132), 
Av(213), Av(231) or Av(312) then C has a rational generating function. 

Proof. Suppose otherwise. Because the class of separable permutations is partially well or- 
dered, its subclasses satisfy the minimum condition of Proposition 2.1, and we can there- 
fore choose among all the counterexamples a minimal class C. We use two properties of C 
repeatedly: 

• all proper subclasses of C have rational generating functions because C is a minimal 
counterexample, and thus 
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• C is atomic because otherwise it would be the union of two strongly rational classes 
and hence strongly rational by Proposition 3.1. 

Our proof that C does not exist begins by ruling out some easy cases. The easiest possibility 
to rule out is when C is either a sum or skew sum of two proper subclasses, which is 
eliminated by Corollary 3.6. 

Next we dispense with the case that C is sum closed (the case that C is skew closed 
is similar). In this case we define C$ as the set of sum indecomposable elements of C. It 
must be the case that C\(C$) = C for otherwise it would be a proper subclass of C and so 
strongly rational; but then, again by Corollary 3.6, the sum closure of C1(C0) would also be 
strongly rational and so its subclass C would be strongly rational, a contradiction. In the 
same way, C1(C0) = C. In other words, every permutation in C is contained in both a sum 
indecomposable permutation and a skew sum indecomposable permutation of C. 

We consider any spine a\, 02, ■ ■ ■ of C (in the sense of Proposition 2.3). Clearly 1 © 
«i, 1 © «2, • • • is also a spine for C because we are assuming that C is sum closed, and since 
C1(C0) = C, each of these permutations is contained in a sum indecomposable element of 
C. However, because C contains only separable permutations, the sum indecomposable 
permutations in C (of length at least 2) are precisely the skew decomposable permutations. 
The only way that 1 © a« can be contained in a sum indecomposable element of C is if it 
embeds completely into one of the skew components of such a permutation. Thus for all i 
either (1 © a;) © 1 or 1 © (1 © ojj) lies in C. As one or the other of these possibilities must 
occur infinitely often, we see that either C = l©CorC = C©l. Proposition 1.3 now shows 
that C contains Av(231) or a symmetry, Av(312), a contradiction. Similarly, we reach a 
contradiction if C is skew closed. 

For the remainder of the proof we may therefore take C to be neither a sum or skew 
sum of two proper subclasses nor to be sum closed or skew closed. To complete the proof 
we shall find a proper subclass U C C for which C C X\U]. This would indeed be a 
contradiction because, by the minimality of C, U would be strongly rational and therefore, 
by Theorem 3.5, X[U] would also be strongly rational. 

We now construct a finite collection of proper subclasses of C whose union will yield 
the desired U. To do this we shall rely on the following characterisation of subclasses of 
X\U], which follows trivially from Proposition 3.4. 

Proposition 4.2. Given classes C and U, we have C C X\U] if and only if, for every -k G C, one of 
the following holds (for nonempty 7 and r): 

• 7T € U, 

• 7T = 7 © r with 7 eW or t £U,or 

• 7r = 7 © r with jeUorreU. 

With the aim of mimicking the structural decomposition provided by this proposition, 
we begin by defining 

C sw = {aeC :a(BCQC}. 
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Note that Csw is a proper subclass of C because C is not sum closed. In fact, Csw is the 
maximum subclass V of C such that V © C C C. Similarly define Catvk maximal such that 
Caw©C = C, Cne maximal such that C®Cne = C, and Cse maximal such that CqCse = C. 
As C is neither sum nor skew sum closed, these are all proper subclasses of C (and may 
indeed be empty). These are the first four classes that will be placed within U. 

Consider any sum decomposable element -k of C and write it = 7 © r (in arbitrary 
fashion). If 7 G Csw or r G Cne then the conditions of Proposition 4.2 are already met. So 
suppose now that we have 7 €" Csw and r ^ Cne with 7 © r G C. Define 

£ 7 = {a G C : 7©o- G C}. 

Clearly £ 7 is a subclass of C, and it is proper because 7 ^ C^vk and thus 7 © C / C. Now 
define 

£> 7 = {cr £C : o-ffi£ 7 CC}, 

where <r © <S 7 denotes {cr©i : 1 G £ 7 }. Again, T> 1 is a subclass of C and it is proper since 
r G £ 7 and r g' Cate. Therefore, since C is not a sum of two proper subclasses, £> 7 © £ 7 is a 
proper subclass of C for all permutations 7 of the type being considered. 

While there may be infinitely many permutations 7 of this type the number of distinct 
classes £> 7 is finite. To see this let B denote the (finite) basis of C and consider the sets 
01(7) n Cl(-B), of which there are but a finite number. Suppose we have two of them which 
happen to be equal, say Cl(7) n Cl(-B) = C1(t") n C1(.B). Now a permutation a fails to lie in 
£ 7 if and only if 7 © a G" C. But this happens if and only if 7 © a contains some /3i © fo G B 
with fa < 7 and /3 2 < a. This means that fa G Cl( 7 ) n Cl{B) = C1(t) n C1(B) and so 
Pi < 7- In turn this implies that 7 © er C and hence a fails to lie in £- also. In other words 
£ 7 = £y. But then 2? 7 = P 7 also. 

Therefore — in addition to Csw> Cse, Cnw and Cne — we include in U the finitely 
many classes D 7 © £ 7 arising from decompositions of this type. By repeating an analogous 
argument for skew decompositions A © 6 with A ^ C^e and ^ C^vy we again find finitely 
many classes, and also include these in U. Since C is atomic, C cannot be equal to a finite 
union of proper subclasses, sosoW 7^ C. This choice of U ensures that the conditions of 
Proposition 4.2 are met, and thus that C C A^[Z/| establishing the desired contradiction, and 
proving the theorem. 

□ 



5. Open problems 

The most obvious question is the converse to our main result: is there a subclass of the 
separable permutations containing Av(231) which has a rational generating function? In 
fact, we are not aware of any finitely based permutation class, separable or otherwise, 
which contains Av(231) and has a rational generating function. (Although we can non- 
constructively prove that there are infinitely based classes satisfying these conditions in a 
manner similar to the proof of Proposition 3.2.) 
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More generally, we are hopeful that the notion of strongly rational classes introduced 
herein will prove interesting and important in future studies of permutation classes. One 
question, inspired by Theorem 3.5, would be: is there a natural characterisation of the 
classes C such that C[U] is strongly rational for all strongly rational classes IA1 A positive 
answer to this question would lend hope to the possibility of a characterisation of the 
strongly rational classes themselves. 
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